With the growing popularity of Internet video and Internet TV, there is an increasing need for streaming solutions that allow a shared synchronized user experience for a group of users. Such synchronized streaming services may be provided to a plurality of users, which are located at different locations. The process for achieving synchronized play-out of two or more receivers that are located at different geographical locations is usually referred to as Inter-Destination Media Synchronization (IDMS) or group synchronization.
IDMS is a synchronization technique that is distinctly different from two other known media synchronization techniques, i.e. intra-stream synchronization for managing the temporal relationship of packets within one stream, e.g. a video stream, that arrives at a receiver; and, inter-stream synchronization (sometimes also referred to as lip-sync) for managing the temporal relationship of packets in two or more related streams, e.g. a video stream and an audio stream, that arrive at a receiver.
IDMS may be used for social TV wherein different receivers at different locations receive and share a media stream (e.g. a live TV program) and wherein the users sharing the media stream may also directly communicate with each other in real-time. IDMS may be used for other purposes, such as synchronization of multiple television outputs in a single physical location, or for the synchronization of different networked speakers throughout a house. For example, a user may wish to watch the same television program on a handheld device in synchronization with a television at home, such that a user may seamlessly switch from one device to another.
Examples of known IDMS systems are described in ETSI TISPAN documents TS 182 027 and 183 063, an article by I. Vaishnavi et al. “From IPTV to shared experiences challenges in design: Distributed media synchronization” (Elsevier Signal Processing: Image Communication 26 (2011) pp. 370-377) and an article by F. Boronat et al. “Multimedia group and inter-stream synchronization techniques: A comparative study” (Elsevier Information Systems 34 (2009) 108-131).
IDMS of a broadcasted (or multicast) stream may be achieved in two steps. First the asynchrony between different receivers is determined by collecting individual status reports from synchronization clients in the receivers. Thereafter, the asynchrony between the receivers is corrected by adjusting the play-out of receivers on the basis of synchronization reports that are sent to the synchronization clients. Adjustment of the play-out may be achieved by delaying the play-out of the media stream at a receiver or transmission of the media stream at the source until the synchronization is restored.
As (live) content is often distributed on the basis of multicast or broadcast, the source is by definition not able to adjust the transmission of the media streams to the different receivers. Therefore, the receivers have to buffer the received data until the play-out time of the content at each receiver is the same. Here, the buffer in the receiver performs two functions. Firstly, buffering is performed to smooth out jitter, allow for loss-repair mechanisms (FEC, RET) and to perform video decoding, wherein packets sometimes depend on future packets. Secondly, buffering is performed for IDMS purposes in order to accommodate for the difference in reception times for different receivers.
Hence, when joining a broadcasted or multicasted stream that is synchronized on the basis of an IDMS system, a synchronization client in receiver first has to wait for data units that can be played-out out before it is able to send a status report on the play-out of the stream to a synchronization server that uses the report for calculating a delay for the joining receiver. After having received the delay information, the joining receiver may delay the play-out of the stream until it is in sync with the other receivers. Hence, the process of joining an IDMS synchronized session may seriously degrade the user experience.
In other applications, further problems may occur. For example, a live interactive classroom for distant learning may use broadcast or multicast to distribute the content and use an IDMS system to synchronize the play-out of the content, Latecomers that would like to join would have to wait for the slowest receiver before starting play-out of the synchronized session. This means that crucial information could be missed, even if the user himself has a high capacity and low-latency network connection with the interactive classroom.
In the article of Montagud et al. “Enhanced adaptive RTCP-based Inter-Destination Multimedia Synchronization approach for distributed applications”, Computer Networks Journal, Vol. 56, Issue 12, pp. 2912-2933, August 2012, a method is described for initiating play-out of a media stream at the same time by a group of receiver (e.g. when starting a scheduled tele-presentation). In that case, all receivers send information about their current clocks to a source in order to calculate the initial (network) delay before the transmission of the media starts. Based on this information, the receivers may fill their buffers until the actual time for play-out has arrived.
Montagud does not address the problem of achieving a synchronized state between a receiver or a group of receivers that is in the process of playing a stream and a receiver that wants to join the streaming session in sync. Furthermore, the solution of Montagud is not suitable for applications such as a live streaming application wherein the system should be able to provide fast or even seamless realization of a synchronized state between a joining receiver and a receiver or a group of receivers that is already playing the media.
Hence, there is a need in the art for improved methods and systems for synchronized data processing between receivers, including synchronized data processing between a receiver that is in the process of processing a stream and a receiver that wants to join the stream.